<?xml version="1.0"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <title>Writing Servers</title>
  </head>

  <body>
    <h1>Writing Servers</h1>

    <h2>Overview</h2>

    <p>This document explains how you can use Twisted to implement
    network protocol parsing and handling for TCP servers (the same
    code can be reused for SSL and Unix socket servers). There is
    a <a href="udp.xhtml">separate document</a> covering UDP.</p>

    <p>Your protocol handling class will usually subclass <code
    class="API">twisted.internet.protocol.Protocol</code>. Most
    protocol handlers inherit either from this class or from one of
    its convenience children. An instance of the protocol class
    is instantiated per-connection, on demand, and will go
    away when the connection is finished. This means that
    persistent configuration is not saved in the
    <code>Protocol</code>.</p>

    <p>The persistent configuration is kept in a <code>Factory</code>
    class, which usually inherits
    from <code class="API">twisted.internet.protocol.Factory</code>. The <code>buildProtocol</code>
    method of the <code>Factory</code> is used to create
    a <code>Protocol</code> for each new connection.</p>

    <p>It is usually useful to be able to offer the same service on
    multiple ports or network addresses. This is why
    the <code>Factory</code> does not listen to connections, and in
    fact does not know anything about the
    network. See <a href="endpoints.xhtml">the endpoints
    documentation</a> for more information,
    or <code class="API">twisted.internet.interfaces.IReactorTCP.listenTCP</code>,
    and the other <code>IReactor*.listen*</code> APIs for the lower
    level APIs that endpoints are based on.</p>

    <p>This document will explain each step of the way.</p>

    <h2>Protocols</h2>

    <p>As mentioned above, this, along with auxiliary classes and
    functions, is where most of the code is. A Twisted protocol
    handles data in an asynchronous manner: the protocol responds
    to events as they arrive from the network; the events arrive as
    calls to methods on the protocol.</p>

    <p>Here is a simple example:</p>
<pre class="python">
from twisted.internet.protocol import Protocol

class Echo(Protocol):

    def dataReceived(self, data):
        self.transport.write(data)
</pre>

    <p>This is one of the simplest protocols. It simply writes back
    whatever is written to it, and does not respond to all events. Here is an
    example of a Protocol responding to another event:</p>
<pre class="python">
from twisted.internet.protocol import Protocol

class QOTD(Protocol):

    def connectionMade(self):
        self.transport.write("An apple a day keeps the doctor away\r\n") 
        self.transport.loseConnection()
</pre>

    <p>This protocol responds to the initial connection with a well
    known quote, and then terminates the connection.</p>

    <p>The connectionMade event is usually where set up of the
    connection object happens, as well as any initial greetings (as
    in the QOTD protocol above, which is actually based on RFC
    865). The <code>connectionLost</code> event is where tearing down of any
    connection-specific objects is done. Here is an example:</p>
<pre class="python">
from twisted.internet.protocol import Protocol

class Echo(Protocol):

    def __init__(self, factory):
        self.factory = factory

    def connectionMade(self):
        self.factory.numProtocols = self.factory.numProtocols+1 
        self.transport.write(
            "Welcome! There are currently %d open connections.\n" %
            (self.factory.numProtocols,))

    def connectionLost(self, reason):
        self.factory.numProtocols = self.factory.numProtocols-1

    def dataReceived(self, data):
        self.transport.write(data)
</pre>

    <p>Here <code>connectionMade</code> and
    <code>connectionLost</code> cooperate to keep a count of the
    active protocols in a shared object, the factory. The factory must
    be passed to <code>Echo.__init__</code> when creating a new
    instance. The factory is used to share state that exists beyond the
    lifetime of any given connection. You will see why this object is
    called a "factory" in the next section.</p>

    <h3>loseConnection() and abortConnection()</h3>

    <p>In the code above, <code>loseConnection</code> is called immediately
    after writing to the transport. The <code>loseConnection</code> call will
    close the connection only when all the data has been written by Twisted
    out to the operating system, so it is safe to use in this case without
    worrying about transport writes being lost. If
    a <a href="producers.xhtml">producer</a> is being used with the
    transport, <code>loseConnection</code> will only close the connection once
    the producer is unregistered.</p>

    <p>In some cases, waiting until all the data is written out is not what we
    want. Due to network failures, or bugs or maliciousness in the other side
    of the connection, data written to the transport may not be deliverable,
    and so even though <code>loseConnection</code> was called the connection
    will not be lost. In these cases, <code>abortConnection</code> can be
    used: it closes the connection immediately, regardless of buffered data
    that is still unwritten in the transport, or producers that are still
    registered. Note that <code>abortConnection</code> is only available in
    Twisted 11.1 and newer.</p>


    <h3>Using the Protocol</h3>

    <p>In this section, you will learn how to run a server which uses your
    <code>Protocol</code>.</p>

    <p>Here is code that will run the QOTD server discussed
    earlier:</p>
<pre class="python">
from twisted.internet.protocol import Factory
from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.internet import reactor

class QOTDFactory(Factory):
    def buildProtocol(self, addr):
        return QOTD()

# 8007 is the port you want to run under. Choose something &gt;1024
endpoint = TCP4ServerEndpoint(reactor, 8007)
endpoint.listen(QOTDFactory())
reactor.run()
</pre>
    <p>In this example, I create a protocol <code class="api"
    base="twisted.internet.protocol">Factory</code>.  I want to tell this
    factory that its job is to build QOTD protocol instances, so I set its
    <code>buildProtocol</code> method to return instances of the QOTD class.  Then, I want to listen
    on a TCP port, so I make a <code>TCP4ServerEndpoint</code> to identify the
    port that I want to bind to, and then pass the factory I just created to
    its <code class="api"
    base="twisted.internet.interfaces.IStreamServerEndpoint">listen</code>
    method.</p>

    <p>Because this is a short example, nothing else has yet started up the
    Twisted reactor.  <code>endpoint.listen</code> tells the reactor to handle
    connections to the endpoint's address using a particular protocol, but the
    reactor needs to be <em>running</em> in order for it to do anything.
    <code>reactor.run()</code> starts the reactor and then waits forever for
    connections to arrive on the port you've specified.</p>

    <p>You can stop the reactor by hitting Control-C in a terminal or calling
    <code>reactor.stop</code>.</p>

    <p>For more information on different ways you can listen for incoming
    connections, see <a href="endpoints.xhtml">the documentation for the
    endpoints API</a>.</p>

    <h3>Helper Protocols</h3>

    <p>Many protocols build upon similar lower-level abstraction.
    The most popular in internet protocols is being line-based.
    Lines are usually terminated with a CR-LF combinations.</p>

    <p>However, quite a few protocols are mixed - they have
    line-based sections and then raw data sections. Examples
    include HTTP/1.1 and the Freenet protocol.</p>

    <p>For those cases, there is the <code>LineReceiver</code>
    protocol. This protocol dispatches to two different event
    handlers - <code>lineReceived</code> and
    <code>rawDataReceived</code>. By default, only
    <code>lineReceived</code> will be called, once for each line.
    However, if <code>setRawMode</code> is called, the protocol
    will call <code>rawDataReceived</code> until
    <code>setLineMode</code> is called, which returns it to using
    <code>lineReceived</code>. It also provides a method,
    <code>sendLine</code>, that writes data to the transport along
    with the delimiter the class uses to split lines (by default,
    <code>\r\n</code>).</p>

    <p>Here is an example for a simple use of the line
    receiver:</p>
<pre class="python">
from twisted.protocols.basic import LineReceiver

class Answer(LineReceiver):

    answers = {'How are you?': 'Fine', None : "I don't know what you mean"}

    def lineReceived(self, line):
        if self.answers.has_key(line):
            self.sendLine(self.answers[line])
        else:
            self.sendLine(self.answers[None])
</pre>

    <p>Note that the delimiter is not part of the line.</p>

    <p>Several other, less popular, helpers exist, such as a
    netstring based protocol and a prefixed-message-length
    protocol.</p>

    <h3>State Machines</h3>

    <p>Many Twisted protocol handlers need to write a state machine
    to record the state they are at. Here are some pieces of advice
    which help to write state machines:</p>

    <ul>
      <li>Don't write big state machines. Prefer to write a state
      machine which deals with one level of abstraction at a
      time.</li>

      <li>Don't mix application-specific code with Protocol
      handling code. When the protocol handler has to make an
      application-specific call, keep it as a method call.</li>
    </ul>

    <h2>Factories</h2>

    <h3>Simpler Protocol Creation</h3>

    <p>For a factory which simply instantiates instances of a
    specific protocol class, there is a simpler way to implement the factory.
    The default implementation of the <code>buildProtocol</code> method calls
    the <code>protocol</code> attribute of the factory to create
    a <code>Protocol</code> instance, and then sets an attribute on it
    called <code>factory</code> which points to the factory
    itself. This lets every <code>Protocol</code> access, and possibly
    modify, the persistent configuration.  Here is an example that uses these
    features instead of overriding <code>buildProtocol</code>:</p>

<pre class="python">
from twisted.internet.protocol import Factory, Protocol
from twisted.internet.endpoints import TCP4ServerEndpoint
from twisted.internet import reactor

class QOTD(Protocol):

    def connectionMade(self):
        # self.factory was set by the factory's default buildProtocol:
        self.transport.write(self.factory.quote + '\r\n')
        self.transport.loseConnection()


class QOTDFactory(Factory):

    # This will be used by the default buildProtocol to create new protocols:
    protocol = QOTD

    def __init__(self, quote=None):
        self.quote = quote or 'An apple a day keeps the doctor away'

endpoint = TCP4ServerEndpoint(reactor, 8007)
endpoint.listen(QOTDFactory("configurable quote"))
reactor.run()
</pre>

    <h3>Factory Startup and Shutdown</h3>

    <p>A Factory has two methods to perform application-specific
    building up and tearing down (since a Factory is frequently
    persisted, it is often not appropriate to do them in <code>__init__</code>
    or <code>__del__</code>, and would frequently be too early or too late).</p>

    <p>Here is an example of a factory which allows its Protocols
    to write to a special log-file:</p>
<pre class="python">
from twisted.internet.protocol import Factory
from twisted.protocols.basic import LineReceiver


class LoggingProtocol(LineReceiver):

    def lineReceived(self, line):
        self.factory.fp.write(line+'\n')


class LogfileFactory(Factory):

    protocol = LoggingProtocol

    def __init__(self, fileName):
        self.file = fileName

    def startFactory(self):
        self.fp = open(self.file, 'a')

    def stopFactory(self):
        self.fp.close()
</pre>

    <h2>Putting it All Together</h2>

    <p>As a final example, here's a simple chat server that allows
    users to choose a username and then communicate with other
    users. It demonstrates the use of shared state in the factory, a
    state machine for each individual protocol, and communication
    between different protocols.</p>

    <a href="listings/servers/chat.py" class="py-listing">chat.py</a>

    <p>The only API you might not be familiar with
    is <code>listenTCP</code>. <code class="API"
    base="twisted.internet.interfaces.IReactorTCP">listenTCP</code> is
    the method which connects a <code>Factory</code> to the network.
    This is the lower-level API
    that <a href="endpoints.xhtml">endpoints</a> wraps for you.</p>

    <p>Here's a sample transcript of a chat session (<em><strong>this</strong></em> is text entered by the user):</p>

<pre class="shell">
$ <em><strong>telnet 127.0.0.1 8123</strong></em>
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
What's your name?
<em><strong>test</strong></em>
Name taken, please choose another.
<em><strong>bob</strong></em>
Welcome, bob!
<em><strong>hello</strong></em>
&lt;alice&gt; hi bob
<em><strong>twisted makes writing servers so easy!</strong></em>
&lt;alice&gt; I couldn't agree more
&lt;carrol&gt; yeah, it's great
</pre>

  </body>
</html>
